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Abstract 

The Enhanced Traffic Situational Awareness on 
the Airport Surface with Indications and Alerts 
(SURF IA) algorithm was evaluated in a fast-time 
batch simulation study at the National Aeronautics 
and Space Administration (NASA) Langley Research 
Center. SURF IA is designed to increase flight crew 
situation awareness of the runway environment and 
facilitate an appropriate and timely response to 
potential conflict situations. The purpose of the study 
was to evaluate the performance of the SURF IA 
algorithm under various runway scenarios, multiple 
levels of conflict detection and resolution (CD&R) 
system equipage, and various levels of horizontal 
position accuracy. This paper gives an overview of 
the SURF IA concept, simulation study, and results. 

Introduction 

Runway incursions are a serious aviation safety 
hazard. As such, the FAA is committed to reducing 
the severity, number, and rate of runway incursions 
by implementing a combination of guidance, 
education, outreach, training, technology, 
infrastructure, and risk identification and mitigation 
initiatives [1], Progress has been made in reducing 
the number of serious incursions - from a high of 67 
in Fiscal Year (FY) 2000 to 6 in FY2010. However, 
the rate of all incursions has risen steadily over recent 
years - from a rate of 12.3 incursions per million 
operations in FY2005 to a rate of 18.9 incursions per 
million operations in FY2010 [1, 2], The National 
Transportation Safety Board (NTSB) also considers 
runway incursions to be a serious aviation safety 
hazard, listing runway incursion prevention as one of 
their most wanted transportation safety improvements 
[3]. The NTSB recommends that immediate warning 
of probable collisions/incursions be given directly to 
flight crews in the cockpit [4], 

The Enhanced Traffic Situational Awareness on 
the Airport Surface with Indications and Alerts 


(SURF IA) application has been established by 
RTCA Special Committee 186 to reduce the 
likelihood and severity of runway incursions and 
collisions. Safety, performance, and interoperability 
requirements (SPR) [5] have been developed for 
SURF IA to increase flight crew situation awareness 
of the runway environment and facilitate an 
appropriate and timely response to potential conflict 
situations. The SURF IA application utilizes cockpit 
display of traffic information (CDTI) to promote 
surface situation awareness and associated flight deck 
indication and alerting concepts for safety assurance. 
The application employs continual own-ship and 
traffic data monitoring and algorithms to detect 
potential conflicts on the runway. Several human-in- 
the-loop studies have been conducted to evaluate the 
SURF IA concept [6, 7, and 8]. 

A fast-time batch simulation study was 
conducted to evaluate the performance of the SURF 
IA algorithm, with variations in surveillance 
accuracy. The algorithm was evaluated under various 
runway scenarios, multiple levels of CD&R system 
equipage, and various levels of horizontal position 
accuracy. Algorithm performance was assessed 
through various metrics including the frequency of 
collisions and near collisions and nuisance and 
missed alerts. This paper presents an overview of the 
SURF IA concept, description of the test method, and 
study results. 

System Description 

Simulation Tool 

A simulation tool, known as Traffic Manager 
(TMX), was utilized for this study. TMX is a desktop 
simulation application designed for interaction 
studies of aircraft in present or future Air Traffic 
Management environments [9]. TMX can serve as a 
stand-alone traffic simulator, scenario generator, 
scenario editor, experiment control station, data 



recording tool, and rapid prototyping environment 
and can operate in real-time or fast-time mode. For 
this study, TMX was used in fast-time mode 
simulating various approach, departure, and taxi 
scenarios at the Chicago O’ Hare International 
(KORD) airport. Although TMX is capable of 
simulating up to 2,000 aircraft simultaneously, only 
two aircraft per scenario were simulated. Each 
aircraft used a six-degree -of-freedom dynamics 
model. 

Some modifications were made to TMX for this 
study. These included: 1) an updated database for the 
KORD airport; 2) creation of a Global Positioning 
System (GPS) sensor model for position accuracy; 3) 
creation of an interface to the CD&R algorithm; 4) 
expansion of the pilot model to handle the required 
taxi and runway maneuvers; and various other minor 
modifications. 

Surveillance Data 

Traffic data are integral to the SURF IA concept. 
For this study, it was assumed that Automatic 
Dependent Surveillance - Broadcast (ADS-B) would 
be used as the means for transmitting (ADS-B Out) 
and receiving (ADS-B In) aircraft surveillance data. 
ADS-B transmissions followed RTCA DO-242A 
specifications [10]; however, some simplifications 
were made to minimize computational overhead, 
such as not implementing models for latency effects 
and transmission line-of-sight and bandwidth 
blockage. For state -vector messages, a 1 Hz data 
transmission rate was specified. The position 
accuracy depended on the GPS measurement errors 
including a bias and an instantaneous jitter. A Gauss- 
Markov process was used to model the time 
correlation between successive position measurement 
errors [11]. 

Navigation Accuracy Category for Position 
(NACp) describes the accuracy of positional 
information. NACp values range from 0 to 11 [10]. 
The horizontal Estimated Position Uncertainty (EPU) 
values for NACp categories of 8 and higher are listed 
in Table 1. 

The Federal Aviation Administration (FAA) has 
issued an ADS-B Out Final Rule [12] which includes 
performance standards for ADS-B Out. The rule 
states that EPU must be less than 0.05 nautical miles 
(nm), which is equivalent to NACp 8. 


Table 1. NACp 


NACp 

95% Horizontal Accuracy Bound (EPU) 

8 

EPU < 92.6 m (0.05 nm, 305.6 ft) 

9 

EPU < 30 m (99 ft) 

10 

EPU < 10 m (33 ft) 

11 

EPU < 3 m (9.9 ft) 


Irrespective of this rule, the SURF IA SPR [5] 
has proposed horizontal position accuracy 
requirements. Through analysis, the SPR identified 
that to meet safety requirements, horizontal position 
accuracy on the aiiport surface needs to be at least 10 
m within 95% containment bounds (NACp 10) to 
allow indications and alerts at virtually all airports in 
the National Airspace System. Validation of these 
requirements is on-going. 

Based on the ADS-B Out Final Rule and SURF 
IA requirements, NACp values of 8, 9, 10, and 1 1 
were evaluated for this study. Truth data, with no 
accuracy errors, was also evaluated. 

Conflict Detection 
SURF IA Algorithm 

SURF IA identifies potential runway conflicts 
that involve aircraft or vehicles in the airport 
maneuvering area and within 3 nm of the runway 
threshold and 1000 ft above field elevation (AFE). 
SURF I A generates both indications and alerts for a 
Cockpit Display of Traffic Information (CDTI). 
SURF IA utilizes traffic surveillance information 
obtained from ADSB-In and generates indications 
and alerts based on the aircraft/vehicle states during 
same runway, very closely spaced parallel runway, 
and intersecting runway operating configurations. Six 
types of aircraft operational states are defined: 1) 

taxiing on a taxiway toward a hold line or stopped at 
a hold line; 2) entering or crossing a runway (not 
lined up with runway); 3) takeoff; 4) approach; 5) 
after landing roll-out on runway (e.g., less than or 
equal to 40 kt); and 6) stopped or taxiing along a 
runway. To prevent inappropriate crew responses 
during departure, indications and alerts are inhibited 
above 80 kt. The SURF IA application does not 
currently address taxiway or low altitude air-to-air 
conflicts, directive alerting, and is not intended for 
use on helicopters or vehicles. A complete 
description of the SURF IA application can be found 
in [5], 




Indications and Alerts 

Indications and alerts notify the flight crew of 
potentially hazardous situations and are presented to 
the flight crew on a CDTI [5]. 

Indications are intended to generate pilot 
awareness and situation assessment by highlighting 
the runway and traffic status as relevant to own-ship 
operations. Indications identify operational 
conditions that are generally normal, yet relevant for 
runway safety and could be a precursor to a non- 
normal situation. Two types of indications are 
defined. 

A traffic indication (TI) highlights a potential 
runway traffic collision/hazard that may emerge in 
the near future. TIs are intended to increase the flight 
crews’ awareness of the relevant runway traffic. 
After a brief assessment of the situation and if 
appropriately cleared, the flight crew may proceed 
with the intended operation. 

A runway status indication (RSI) identifies if the 
runway that own-ship is approaching or using is in- 
use or occupied by other traffic and is not suitable for 
entering, takeoff, or landing. Before proceeding, the 
crew should ensure they have the appropriate 
clearance and the indicated traffic is not a factor. 

Alerts identify potential collision hazards which 
may require timely response by the flight crew to 
avoid a collision. Alerts have priority over 
indications. Auditory and visual annunciations are 
required for alerts. A two-level alerting scheme is 
defined. 

Caution alerts are generated for conditions that 
require immediate flight crew awareness and 

subsequent flight crew response. Generally, caution 
alerts are generated with sufficient time to evaluate 
the situation to be prepared to respond. 

Warning alerts are generated for conditions that 
require immediate flight crew awareness and 

immediate flight crew response. Warning alerts may 
occur without preceding caution alerts. 

Test Method 

Data collection occurred for runway conflict 
scenarios. Only two aircraft were included in each 
scenario to limit the interaction in this initial fast- 
time study. For ease of discussion, the aircraft will be 
referred to as Aircraft A and Aircraft B . 


CD&R Equipage 

Various levels of CD&R system equipage were 
simulated for this study: a) both aircraft equipped; b) 
neither aircraft equipped; or, c) one or the other 
aircraft equipped. 

When an aircraft was not equipped, it would 
follow its planned flight path to the end of the test 
run. When an aircraft was equipped, it would take 
action after a warning alert was generated by 
following an appropriate maneuver (e.g. go-around, 
abort, stop), depending on the operational phase. The 
maneuver was based on the relative location of the 
aircraft at the projected closest point of approach 
(CPA). 

Test Scenarios 

Seven runway scenarios were developed. 
Variability was introduced into the scenarios by 
varying the location of the aircraft, speed of the 
aircraft, and/or time when the aircraft started to 
proceed along its predefined route. As a result, not 
every test run resulted in a conflict or collision. Due 
to space limitations, only two scenarios are reported 
in this document. 

Runway Scenario - Arrival with taxi crossing 

This scenario evaluated the situation where an 
aircraft was on approach when another aircraft taxied 
perpendicular across the runway. 

The initial condition (IC) for Aircraft A was at 
3.5 nm from the threshold at 1 1 10 ft above field level 
(AFL) at an indicated airspeed of 138 kt. Aircraft A 
flew at a constant altitude to intercept the glideslope 
at 3.3 nm from the threshold and then descended on a 
3 degree glidepath for a straight-in approach to 
Runway 10. For the nominal flight plan, Aircraft A 
landed, decelerated at 2.5 m/s, taxied down the 
runway at 30 kt, and then slowed to exit the runway 
at Taxiway M7 at 3 kt. Aircraft B started at various 
locations around Runway 10 and taxied across 
stalling from a complete stop and accelerating at 1 
m/s/s to 15 kt. The actual taxiways for KORD were 
not used in this study. Instead, Aircraft B’s initial 
position was placed at 14 different locations along 
the length of Runway 10 (0, 1000, 1500, 2000, 2500, 
3000, 3500, 4000, 4500, 5000, 6000, 7500, 9000, and 
10,000 feet from the runway threshold) simulating 
various taxiway entry points and at 1 8 locations away 
from the runway (300, 320, 340, 360, 380, 400, 450, 



500, 550, 600, 650, 700, 800, 900, 1,000, 1,200, 
1,400, and 1,600 feet from the runway centerline) - 
see Figure 1. Aircraft B (red chevron, Figure 1) 
began to taxi when Aircraft A (blue chevron, Figure 
1) was at various points in its approach and rollout 
(from 3.5 nm to the glidepath intercept on the 
runway, at 0.5 nm intervals, crossing the runway 
threshold, glidepath aimpoint (1000 ft), and 3000 ft, 
4500 ft, 1 nm, 8000 ft, and 9000 ft past the 
threshold). If maneuvering were required based on a 
CD&R warning alert being triggered in an 
appropriately equipped aircraft, Aircraft A would 
conduct a go-around if above 70 ft AFL; otherwise, it 
would continue to land and stop on the runway. 
Aircraft B, if equipped, would stop if its nose had not 
reached the runway shoulder (greater than 100 ft 
from the runway centerline) at the projected stopping 
point; otherwise, it would continue to taxi across the 
runway. When conducting an emergency stop by 
either aircraft, a 4 m/s/s deceleration rate was used. 

Runway Scenario - Departure with taxi crossing 

This scenario tested the situation where an 
aircraft was on departure when another aircraft taxied 
perpendicular across the runway. 

The configuration for this scenario was 
similar to the arrival with taxi crossing scenario, 
except the IC for Aircraft A was in position on 
Runway 10 for departure. Aircraft B’s 1C was placed 
at 12 different locations along the length of Runway 
10 (0, 60, 280, 660, 1,100, 1,800, 2,500, 3,400, 5,200, 
6,800, 8,000, and 9,000 feet from the runway 
threshold) and at the same 18 locations away from 
the runway as in the previous scenario - see Figure 2. 
Aircraft B began to taxi when Aircraft A was at 
various speeds and locations along its departure and 
climb out (0, 20, 40, 60, 80, 100, 120, and 140 kt, 
liftoff, and 100, 300, and 500 ft AFL). If 
maneuvering were required based on a warning alert 
being triggered in an appropriately equipped aircraft, 
Aircraft A would abort departure if below takeoff 
decision speed (131 kt) and stop on the runway; 
otherwise it would continue departure. Aircraft B, if 
equipped, would maneuver as described in the 
previous scenario. 

Pilot Reaction Delay 

A delay was incorporated to simulate the 
reaction time from when a pilot would receive a 
warning alert until action was taken to resolve the 


situation. The following delay times were used for 
this study: 5 seconds (sec) when aircraft was on 
approach, 3 sec when aircraft was rolling out, 2 sec 
during taxi, and 2 sec during departure. These delay 
times were selected based on reaction delays 
experienced during previous piloted simulation and 
flight testing. 

Test Matrix 

Algorithm performance was evaluated for the 
conflict scenarios described above using the SURF 
IA CD&R algorithm for five levels of surveillance 
accuracy (NACp of 8, 9, 10, 11, and truth), and four 
levels of CD&R system equipage (neither aircraft 
equipped, only Aircraft A equipped, only Aircraft B 
equipped, and both aircraft equipped). Thus, 20 cases 
were examined for each of the scenarios. 

The number of replicates for each treatment 
combination varied according to level of surveillance 
accuracy. As shown in Table 2, more replicates were 
conducted for the lower NACp values in order to 
increase the level of precision for estimating the true 
location of the aircraft. 


Table 2. Number of Replicates 


NACp 

Number of replicates 

8 

7 

9 

6 

10 

4 

11 

3 

Truth 

1 


Some of the test conditions from Figures 1 and 
2, in which a conflict would obviously not occur, 
were omitted in order to reduce the size of the test 
matrix. 

Data were collected for the test runs in random 
order by scenario. For data manageability of the 
scenarios that required a large number of test runs, 
data were collected randomly grouped by level of 
surveillance accuracy and CD&R system equipage. 

Test Metrics 

Some of the metrics utilized for this study are 
defined in this section. All data is referenced from the 
aircraft center-of-gravity (CG), unless noted 
otherwise. 
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Figure 1. Runway Scenario - Arrival with Taxi Crossing Initial Conditions 
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Figure 2. Runway Scenario - Departure with Taxi Crossing Initial Conditions 


Near-Collision / Collision 

A near-collision was counted if the CG’s of the 
two aircraft were < 300 ft apart laterally and vertical 
separation was < 200 ft. A collision was counted if 
the aircraft CG’s were < 150 ft apart laterally and 
vertical separation was < 100 ft. 

Nuisance / Missed Indications and Alerts 

According to the SURF IA SPR [5], a nuisance 
indication or alert is defined as any indication or alert 
generated by a properly functioning CD&R system 
that is inappropriate or unnecessary for the particular 
situation. Nuisance indications and alerts could 
distract the flight crew unnecessarily, reduce 
confidence in the system, and negatively affect safety 
and operational effectiveness. Repeated nuisance 
indications and alerts could decrease the use of 
CD&R and reduce expeditious flight crew response 
to true indications and alerts. 

A missed indication or alert is defined as a 
failure to provide an indication or alert when it is 
necessary provided own-ship and traffic are 
adequately equipped [5]. Missed indications and 
alerts represent a reduction in CD&R benefits. 

The SURF IA SPR [5] definitions for nuisance 
and missed boundaries were applied. Horizontal 
position error was the only source of error modeled. 
Other sources of error, such as vertical position error, 
airport database error and flight technical error, were 
not included. 

The following nuisance and missed boundary 
definitions applied for Aircraft A: 

• When the aircraft was on approach, an approach 
corridor as defined in the SURF IA SPR for 
NACp 8 with a probability of missed alert of 
0.01 was used since NACp 8 and higher was 
being evaluated. The corridor width was +/- 
321.5 ft at the runway threshold and linearly 
increased to +/- 964.6 ft at 3 nm away from the 
runway threshold. The nuisance boundary 
definition was when the true aircraft position 
was outside the approach corridor, but the 
detected position was within the approach 
corridor. The missed boundary definition was 
when the aircraft’s true position was within the 
approach corridor, but the detected position was 
outside the approach corridor. Since the true 
position of the approach aircraft tracked the 


extended runway centerline, the aircraft could 
never enter the nuisance boundary. 

• When the aircraft had crossed the runway 
threshold on landing or was traveling along a 
runway, the nuisance boundary definition was 
when the aircraft’s true position was farther than 
one runway width (150 ft) from the runway 
centerline, but the detected position was within 
one runway width of the centerline. The missed 
boundary definition was when the aircraft’s true 
position was within one runway width of the 
runway centerline, but the detected position was 
greater than one runway width from the 
centerline. 

The following nuisance and missed boundary 
definitions applied for Aircraft B: 

• When the aircraft was taxiing across the runway, 
the nuisance boundary definition was when the 
true position of the aircraft’s nose (when 
entering) or tail (when exiting) was at or behind 
the hold line, but any part of the detected aircraft 
(from nose to tail) was between the runway 
shoulder edges. The missed boundary definition 
was when the true position of any part of the 
aircraft was between the runway shoulder edges, 
but the detected nose position (when entering) 
or tail position (when exiting) was outside of the 
runway shoulder edges. A shoulder width of 7.5 
m (25 ft), as defined in the SURF IA SPR for 
Aerodrome Code 4, was used. A 150 ft wide 
runway was assumed; therefore, the distance 
between shoulder edges was approximately 200 
ft. The hold line was located 225 ft from the 
runway centerline. 

Since only true vertical position was used, there 
was no opportunity for a nuisance or missed 
condition in the vertical direction. 

An indication or alert was considered to be a 
nuisance if the indication or alert was generated when 
the aircraft was within a nuisance boundary, based on 
the definitions above. 

A straight-forward corollary for a missed alert 
definition does not exist. If the aircraft was within the 
missed boundary, based on the definitions above, and 
an alert was not generated, that did not necessarily 
mean that an alert should have been generated. Even 
though one of the aircraft was in the missed 
boundary, the geometry of the aircrafts’ trajectory 



may not be on a collision path. Therefore, if an alert 
was generated when using truth data but an alert was 
not generated at the same instance when using NACp 
data, then a missed alert was counted. This definition 
is algorithm dependent. 

Results 

A summary of quantitative results is presented. 
All data is referenced from the aircraft CG, unless 
noted otherwise. For the aircraft used in this study, 
the nose position was 72.8 ft from the CG and the tail 
position was 82 ft from the CG. 

For each scenario, the data analysis was limited 
to the area of interest, i.e., until the aircraft reached 
the CPA or until 10 seconds after a warning alert 
terminated, whichever was later. Also, both aircraft 
broadcasted the same level of positional accuracy for 
each test run. 

The Kruskal- Wallis test was utilized to provide a 
non-parametric statistical hypothesis test to detect 
differences in NACp accuracy and CD&R equipage 
using a significance level of a = 0.05 and N = number 
of test runs. When statistically significant differences 
were detected, post-hoc analysis was conducted using 
the non-parametric Mann- Whitney test. 

Runway Scenario - Arrival with taxi crossing 

For each of the 20 cases in this scenario, 2,367 
combinations of the initiation delay and initial 
position for Aircraft B (taxiing aircraft) were 
evaluated, for a total of 198,828 test runs. 

Algorithm performance - The data (Table 3) 
shows that, for Aircraft A, TIs were generated on 
20% to 57% of the test runs and RSIs were issued on 
approximately 30% of runs. For Aircraft B, TIs were 
generated on 57% to 81% of runs and RSIs were 
issued on approximately 75% of runs. Caution alerts 
were generated on approximately 24% of the runs 
and warning alerts were issued on approximately 
37% of the runs for either aircraft, almost 
independent of the NACp levels. The number of TIs 
for NACp 8 accuracy was statistically different from 
NACp 9, 10, 11, and truth accuracies over all 
equipage levels. There was no statistical difference 
between accuracy levels for the number of RSIs, 
caution, and warning alerts over all equipage levels. 


Toggling occurred when multiple instances of 
indications and alerts were generated during a test 
run. Toggling is undesirable (i.e., it is a distraction to 
the flight crew and would cause mistrust in the 
technology). 

Toggling occurred frequently for NACp 8 
accuracy (see Table 3). The number of multiple TIs, 
RSIs, and caution and warning alerts for NACp 8 
accuracy was statistically different from NACp 9, 10, 
1 1 and truth accuracies over all equipage levels. This 
toggling included gaps between indications and alerts 
in many instances. 

For Aircraft B, toggling occurred for indications 
when using true position data. In some instances this 
occurred by design. Some situations warrant a TI; as 
the situation progresses, an RSI or alert is generated, 
then, if the situation changes, the indication may be 
degraded back to a less severe TI. It was determined 
that some of the multiple indications occurred as 
Aircraft B taxied across an intersecting runway prior 
to crossing Runway 10. A TI was issued as the 
aircraft was taxiing toward Runway 10; however, 
before reaching Runway 10, the aircraft crossed an 
intersecting runway. Since indications are not 
required in that situation, the indication was no 
longer issued. After crossing the intersecting runway, 
however, the TI was issued again as the aircraft 
approached Runway 10. The SURF IA algorithm 
does not have any mechanisms in place to address 
toggling between aircraft states. 

Missed and nuisance alerts - The number of 
runs in which the aircraft entered the defined missed 
and nuisance boundaries increased as the position 
accuracy decreased, as shown in Table 4. Aircraft can 
cross into the missed and nuisance boundary multiple 
times throughout a test run, for varying lengths of 
time. The number of times (count) and amount of 
time (duration and percentage of run length) that the 
aircraft were within the boundaries was generally 
greater when using less accurate data (see Table 4). 
However, there was no statistical difference between 
accuracy levels when analyzing the number of times 
entering the missed and nuisance boundary over all 
equipage levels. 

For the approach aircraft (Aircraft A), the 
majority of occurrences of entering the missed 
boundary was after the aircraft had crossed the 
runway threshold for landing. The aircraft only 
entered the missed boundary while on approach 



Table 3. Indication and Alert Statistics by Aircraft for Arrival / Taxi Crossing Runway Scenario 


NACp 

Total # 
Runs 

TI 

(% runs) 

Multiple 

TI 

(% runs) 

RSI 

(% runs) 

Multiple 

RSI 

(% runs) 

Caution 
Alerts 
(% runs) 

Multiple 
Caution 
Alerts 
(% runs) 

Warning 
Alerts 
(% runs) 

Multiple 
Warning 
Alerts 
(% runs) 

Aircraft A 

8 

66,276 

57.0 

17.6 

32.5 

5.8 

26.0 

2.5 

33.4 

9.8 

9 

56,808 

28.0 

1.3 

31.3 

1.1 

24.0 

0.0* 

37.4 

1.8 

10 

37,872 

21.2 

0.0 

31.5 

0.0 

24.1 

0.0 

37.5 

0.0 

11 

28,404 

19.4 

0.0 

29.0 

0.0 

24.0 

0.0 

37.3 

0.0 

Truth 

9,468 

21.1 

0.0 

31.7 

0.0 

24.0 

0.0 

37.3 

0.0 

Aircraft B 

8 

66,276 

81.0 

41.1 

78.6 

36.7 

26.0 

5.1 

33.6 

11.6 

9 

56,808 

67.9 

13.1 

79.5 

11.0 

24.0 

0.1 

37.6 

2.1 

10 

37,872 

63.3 

6.8 

79.1 

3.6 

24.2 

0.0 

37.9 

0.0 

11 

28,404 

57.2 

6.2 

72.5 

2.7 

24.2 

0.0 

37.8 

0.0 

Truth 

9,468 

61.7 

5.8 

78.8 

2.1 

24.1 

0.0 

37.8 

0.0 


* Only four occurrences 


Table 4. Missed and Nuisance Boundary Statistics by Aircraft for Arrival / Taxi Crossing Runway 

Scenario 


NACp 

Entered Missed Boundary 

Entered Nuisance Boundary 

# runs, 
% runs 

Count 
(weighted 
mean, SD) 

Duration 
(seconds) 
(mean, SD) 

%of 

Run 

Length 

# runs, 
% runs 

Count 
(weighted 
mean, SD) 

Duration 
(seconds) 
(mean, SD) 

%of 

Run 

Length 

Aircraft 7 


8 

54669, 82.5 

6.9, 4.2 

11.7, 11.9 

7.9 

799, 1.2 

1.0, 1.0 

0.6, 0.2 

0.3 

9 

744, 1.3 

1.5, 1.0 

1.5, 1.6 

0.8 

395, 0.7 

1.0, 1.0 

0.6, 0.2 

0.2 

10 

224, 0.6 

1.0, 0.0 

O 

La 

p 

Lo 

0.2 

231, 0.6 

1.0, 0.0 

0.4, 0.2 

0.2 

11 

116, 0.4 

1.0, 0.0 

0.2, 0.1 

0.1 

104, 0.4 

1.0, 0.0 

0.2, 0.1 

0.1 

Truth 

0, 0.0 

o 

b 

o 

b 

o 

b 

o 

b 

0.0 

0, 0.0 

o 

b 

o 

b 

o 

b 

o 

b 

0.0 

Aircraft I 

5 

8 

54627, 82.4 

2.1, 1.6 

4.1, 2.8 

3.6 

19692, 29.7 

3.6, 1.7 

6.7, 10.6 

5.3 

9 

42329, 74.5 

1.3, 1.1 

1.5, 1.0 

1.3 

58, 0.1 

1.6, 0.8 

1.3, 2.0 

1.2 

10 

25011, 66.0 

1.1, 1.0 

o 

bs 

p 

0.5 

0, 0.0 

o 

b 

o 

b 

o 

b 

o 

b 

0.0 

11 

12223, 43.0 

1.1, 0.9 

0.3, 0.1 

0.2 

0, 0.0 

o 

b 

o 

b 

o 

b 

o 

b 

0.0 

Truth 

0, 0.0 

o 

b 

o 

b 

o 

b 

o 

b 

0.0 

0, 0.0 

o 

b 

o 

b 

o 

b 

o 

b 

0.0 


before crossing the runway threshold during 2.8% of 
the test runs when using NACp 8 accuracy. Since 
Aircraft A tracked the extended centerline on 
approach and centerline after landing, the nuisance 
boundary was entered as the aircraft was exiting the 
runway. 

The taxiing aircraft (Aircraft B) entered the 
missed boundary at least once for a high percentage 
of the test runs for accuracy levels of NACp 8 to 11. 


This was due to the criteria for entering the missed 
boundary. The aircraft was counted as entering the 
missed boundary when the true position of any part 
of the aircraft was determined to be between the 
runway shoulder edges, but the detected nose 
position (when entering) or tail position (when 
exiting) was outside of the runway shoulder edges. 
There was no buffer between when the aircraft was 
inside or outside the missed boundary so measurable 





difference between the true and detected position 
could cause a missed boundary to be counted. 

The number of test runs that contained missed 
and nuisance indications and alerts were relatively 
low, overall, as shown in Table 5. The missed 
indication and alert definition, as noted previously, is 
algorithm dependent. Missed indications and alerts 
for both aircraft were highest when using NACp 8 
accuracy. There was no statistical difference between 
accuracy levels for the number of missed TIs, RSIs, 
and caution alerts over all equipage levels. The 
number of missed warning alerts for NACp 8 
accuracy was statistically different from NACp 9, 10, 
11, and truth accuracies over all equipage levels. For 
Aircraft B only, the number of nuisance TIs, RSIs, 
and caution and warning alerts for NACp 8 accuracy 
was statistically different from NACp 9, 10, 11, and 
truth accuracies over all equipage levels. 

Aircraft B did, however, experience missed TIs 
under the truth accuracy condition. This unexpected 
event occurred because even though the aircraft were 
broadcasting true position data, the ADS-B 
transmission model was still being used. The 
transmission model resulted in a slight delay between 
the aircraft’s position at the time of transmitting the 
ADS-B message and the position at the time of 
reception of the ADS-B message by Aircraft B. This 


delay was present in all scenarios, but this position 
difference was negligible compared to the NACp 
position uncertainty error. In the missed TI scenarios, 
Aircraft B did not detect a potential conflict with 
Aircraft A based on the broadcast position, but if 
instantaneous position information were used for 
Aircraft A, a potential conflict would have been 
detected. The small error introduced by the 
movement of Aircraft A between transmission and 
reception of the ADS-B message resulted in just 
enough difference in relation to Aircraft B’s position 
to result in the missed alerts. 

For Aircraft A, all of the missed indications and 
alerts, for all accuracy levels, occurred after the 
aircraft had crossed the runway threshold for landing. 
An indication was considered a nuisance if it was 
generated at the same time the aircraft was 
determined to be within the nuisance boundary. 
Therefore, nuisance indications for Aircraft A only 
occurred as the aircraft was exiting the runway (see 
explanation above). 

Unnecessary maneuvering - Previous research 
has shown that pilots instinctively react upon 
receiving airport traffic warning alerts in the flight 
deck [13] without necessarily confirming with 
secondary or additional information first. It is critical 
that alerting only occurs when needed; otherwise, 


Table 5. Missed and Nuisance Indication and Alert Statistics by Aircraft for Arrival / Taxi Crossing 

Runway Scenario 



* Negligible number of occurrences 
+ 12 occurrences 




these unnecessary maneuvers can cause delays, 
equipment wear, and other costs to airlines. 

To evaluate this situation, maneuvering was 
considered unnecessary if made based on a warning 
alert issued when the aircraft were broadcasting 
NACp accuracy, but for the same test conditions, a 
warning alert was not issued when broadcasting true 
position data. This measure quantifies untimely 
nuisance alerts using an algorithm-dependent 
methodology. 

The percentage of test runs in which the aircraft 
maneuvered unnecessarily when using NACp data 
accuracy is shown in Table 6. Thus, as the accuracy 
decreased, the frequency of occurrences of 
unnecessary maneuvers increased. With a NACp 
value of 8, approximately 7% or 7 in 100 of the 
warning alerts were unnecessary. With a NACp value 
of 11, approximately 1 in 1000 warning alerts were 
unnecessary for Aircraft A and 5 in 1000 were 
unnecessary for Aircraft B. The number of 
unnecessary maneuvers for NACp 8 accuracy was 
statistically different from NACp 9, 10, and 1 1 
accuracies for both Aircraft A and Aircraft B. 

Table 6. Unnecessary Maneuvers by Aircraft 
during Arrival / Taxi Crossing Runway Scenario 


NACp 

Aircraft A 
(% runs) 

Aircraft B 
(% runs) 

8 

7.4 

7.5 

9 

1.2 

1.4 

10 

0.5 

0.9 

11 

0.1 

0.5 


Collision avoidance - By the design of the 
scenarios, approximately 20% of the runs resulted in 
a near collision (NC) and approximately 10% 
resulted in a collision (C) in the absence of CD&R, as 
shown in Table 7. The addition of CD&R did not 
have much effect on collision avoidance with 
approximately 18% of the runs resulting in near 
collision and approximately 8% in collisions when 
both were equipped. Collision avoidance was not 
significantly affected by the CD&R system equipage 
levels. 

For the taxiing aircraft (Aircraft B), a warning 
alert was not timely. The warning generally did not 
occur until the aircraft was on or almost on the 
runway (i.e., the mean, true position of Aircraft B 


when it received the warning alert was 107.6 ft, 
standard deviation (SD) 59.9 ft, from the runway 
centerline). Since the aircraft was already past the 
runway shoulder, no action was taken and the aircraft 
continued across the runway (as per the design of the 
test maneuvers). 

Some collisions were also unavoidable since the 
approach aircraft (Aircraft A) was issued a warning 
alert during a critical operational phase. Most often, 
the warning alert occurred during high speed rollout 
without enough separation from the traffic to stop in 
time to avoid a collision. Other times the warning 
occurred when the aircraft was too close to the 
ground to go around and the collision occurred as the 
aircraft continued along its predetermined path. 
Some collisions also occurred during low speed (> 40 
kt) taxi on the runway or as the aircraft was exiting 
the runway - conditions for which the SURF IA 
CD&R algorithm does not issue warning alerts; 
therefore, no avoidance action was taken. 

Collision avoidance was not significantly 
affected by the horizontal position accuracy level; 
however, there were slightly less collisions when 
using NACp 8 accuracy. In some instances, the 
warning alert was generated when the sensed aircraft 
was on the runway, but the actual (true) location of 
the aircraft was far enough back from the runway that 
it could come to a complete stop before actually 
reaching the runway shoulder. 

Runway Scenario - Departure with taxi 
crossing 

For each of the 20 cases in this scenario, 1,077 
combinations of the initiation delay and initial 
position for Aircraft B were evaluated, for a total of 
90,468 test runs. 

Algorithm performance - The data (Table 8) 
shows that, for Aircraft A, TIs were generated on 0% 
to 50% of the test runs and RSIs were issued on 
approximately 3% or less of runs. For Aircraft B, TIs 
were generated on 60% to 87% of runs and RSIs 
were issued on approximately 77% of runs. Warning 
alerts were issued on approximately 30% of the runs 
for either aircraft, almost independent of the NACp 
levels. Caution alerts were not generated by either 
aircraft for this scenario. The number of TIs for 
NACp 8 accuracy was statistically different from 
NACp 9, 10, 11, and truth accuracies over all 




Table 7. Percentage of Near- Collisions (NC) and Collisions (C) for Equipage Combinations for Arrival / 

Taxi Crossing Runway Scenario 


NACp 

# Runs per 
Equipage 

Neither 

Aircraft A 

Aircraft B 

Both 

NC 

C 

NC 

C 

NC 

C 

NC 

C 

8 

16,569 

19.3 

9.3 

18.7 

8.9 

18.9 

7.9 

17.9 

8.0 

9 

14,202 

19.3 

9.3 

19.0 

9.2 

18.7 

9.0 

18.5 

8.7 

10 

9,468 

19.3 

9.3 

19.0 

9.2 

18.6 

8.9 

18.4 

OO 

o o 

11 

7,101 

19.3 

9.3 

19.0 

9.0 

18.7 

8.9 

18.4 

OO 

OO 

Truth 

2,367 

19.3 

9.3 

19.1 

9.1 

18.6 

9.0 

18.4 

8.7 


Table 8. Indication and Alert Statistics by Aircraft for Departure / Taxi Crossing Runway Scenario 


NACp 

Total # 
Runs 

TI 

(% runs) 

Multiple TI 

(% runs) 

RSI 

(% runs) 

Multiple RSI 
(% runs) 

Warning Alerts 
(% runs) 

Multiple Warning 
Alerts 
(% runs) 

Aircrai 

ft A 

8 

30,156 

49.8 

14.4 

3.3 

1.0 

24.9 

6.3 

9 

25,848 

7.4 

1.0 

1.1 

0.1 

31.1 

1.1 

10 

17,232 

0.0 

0.0 

0.6 

0.0 

31.7 

0.0 

11 

12,924 

0.0 

0.0 

0.7 

0.0 

31.8 

0.0 

Truth 

4,308 

0.0 

0.0 

0.7 

0.0 

31.7 

0.0 

Aircral 

ft B 

8 

30,156 

87.1 

70.7 

66.0 

34.9 

24.3 

7.5 

9 

25,848 

67.8 

14.2 

77.8 

9.2 

30.2 

1.0 

10 

17,232 

61.2 

0.8 

77.8 

0.6 

30.8 

0.0 

11 

12,924 

60.1 

0.8 

77.7 

0.1 

30.9 

0.0 

Truth 

4,308 

59.9 

0.7 

77.7 

0.0 

30.7 

0.0 


equipage levels. There was no statistical difference 
between accuracy levels for the number of RSIs and 
warning alerts over all equipage levels. 

As the position accuracy was reduced, indication 
and alert toggling occurred more frequently, 
particularly for NACp 8 and 9 accuracies (see Table 
8). For Aircraft A, the number of multiple TIs and 
warning alerts for NACp 8 accuracy was statistically 
different from NACp 9, 10, 11, and truth accuracies 
over all equipage levels. There was no statistical 
difference between accuracy levels for number of 
multiple RSIs over all equipage levels. For Aircraft 
B, the number of multiple TIs and RSIs for NACp 8 
accuracy was statistically different from NACp 9 and 
NACp 9 was statistically different from NACp 10, 
11, and truth accuracies over all equipage levels. The 
number of multiple warning alerts for NACp 8 
accuracy was statistically different from NACp 9, 10, 


1 1 and truth accuracies over all equipage levels. The 
toggling included gaps between alerts in many 
instances. In addition to position accuracy, the 
toggling can also be a result of aircraft collision 
avoidance maneuvering. 

For Aircraft B, TI toggling occurred when 
accurate (true) position data was transmitted. This 
toggling occurred by design. A TI was initially issued 
for Aircraft B and as the situation progressed, an RSI 
and warning alert were issued, then, after Aircraft A 
rejected the departure, a TI was issued again. 

The SURF IA SPR [5] specifies that indications 
and alerts must be inhibited above 80 kt. But as 
currently implemented, the SURF IA algorithm 
calculates alerts throughout the departure when the 
aircraft is traveling greater than 80 kt (the mean 
ground speed when a warning alert was generated on 





Aircraft A was 121.7 kt. SD 36.6 for truth accuracy). 
These data were included in Table 8 even though 
these alerts would not be displayed in the cockpit as 
per the SPR. For the test runs in which the departing 
aircraft was to take action (Aircraft A only and both 
equipped), warning alerts were generated when the 
aircraft was traveling less than 80 kt in only 5 % to 
7 % of the test runs; without this restriction, warning 
alerts were generated on 23 % to 32 % of the test runs 
(see Table 9 ). More research is necessary to 
determine the collision avoidance benefits of alerting 
after 80 kt versus the risk of pilots making 
inappropriate responses at high speed. 


Table 9. Warning Alert Data for Departure 
Aircraft 


NACp 

Warning Alerts 
(% runs) 

Warning Alerts 
when < 80 kt 
(% runs) 

8 

23.4 

7.0 

9 

32.5 

5.9 

10 

31.7 

5.2 

11 

31.8 

5.3 

Truth 

29.6 

5.2 


Missed and nuisance alerts - The number of 
runs in which the aircraft entered the defined missed 
and nuisance boundaries increased as the position 
accuracy decreased, as shown in Table 10 . The 
number of times (count) and amount of time 


(duration and percentage of run length) that the 
aircraft were within the boundaries was greater when 
using less accurate data (see Table 10 ). This was 
particularly true when using NACp 8 accuracy. The 
number of times Aircraft A entered the missed 
boundary for NACp 8 accuracy was statistically 
different from NACp 9 , 10 , 11 , and truth accuracies 
over all equipage levels. There were no instances of 
Aircraft A entering the nuisance boundary. For 
Aircraft B, there was no statistical difference between 
NACp accuracy levels when analyzing the number of 
times entering the missed boundary over all equipage 
levels. The number of times Aircraft B entered the 
nuisance boundary for NACp 8 accuracy was 
statistically different from NACp 9 , 10 , 11 , and truth 
accuracies over all equipage levels. 

The departing aircraft (Aircraft A) entered the 
missed boundary along its entire departure path. 
Since Aircraft A tracked the runway centerline on 
departure, it was not possible for the aircraft to enter 
the nuisance boundary. 

The taxiing aircraft (Aircraft B) entered the 
missed boundary at least once for a high percentage 
of the test runs for accuracy levels of NACp 8 to 11 . 
This was due to the criteria for entering the missed 
boundary. The aircraft was counted as entering the 
missed boundary when the true position of any part 
of the aircraft was determined to be between the 
runway shoulder edges, but the detected nose 
position (when entering) or tail position (when 


Table 10. Missed and Nuisance Boundary Statistics by Aircraft for Departure / Taxi Crossing Runway 

Scenario 



Entered Missed Boundary 

Entered Nuisance Boundary 


# runs. 

Count 

Duration 

%of 

# runs, 

Count 

Duration 

%of 


% runs 

(weighted 

(seconds) 

Run 

% runs 

(weighted 

(seconds) 

Run 

NACp 


mean, SD) 

(mean, SD) 

Length 


mean, SD) 

(mean, SD) 

Length 

Aircraft A 

8 

24675 , 81.8 

4 . 4 , 

3.5 

9 . 0 , 

8.4 

23.7 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

9 

48 , 0.2 

2.1, 

1.2 

2.1, 

2.5 

5.0 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

10 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

11 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

Truth 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

Aircraft I 

1 

8 

18392 , 61.0 

1.6, 

1.2 

3 . 6 , 

2.6 

9.6 

5946 , 19.7 

2 . 4 , 

1.4 

4 . 2 , 

7.8 

9.2 

9 

11479 , 44.4 

1.1, 

1.0 

1 . 3 , 

0.9 

3.8 

8, 0.1 

1.1, 

0.8 

0 . 3 , 

0.2 

0.9 

10 

6419 , 37.3 

1.0, 

1.0 

0 . 5 , 

0.3 

1.5 

0, 0.0 

0.0, 

0.0 

0, 

0.0 

0.0 

11 

2776 , 21.5 

1.0, 

1.0 

0 . 3 , 

0.1 

0.8 

0, 0.0 

0.0, 

0.0 

0, 

0.0 

0.0 

Truth 

0, 0.0 

0.0, 

0.0 

0.0, 

0.0 

0.0 

0, 0.0 

0.0, 

0.0 

0, 

0.0 

0.0 





Table 11. Missed and Nuisance Indication and Alert Statistics by Aircraft for Departure / Taxi Crossing 

Runway Scenario 


NACp 

Total # 
Runs 

Missed 

Nuisance 

TI 

(% runs) 

RSI 

(% runs) 

Caution 
Alert 
(% runs) 

Warning 
Alert 
(% runs) 

TI 

(% runs) 

RSI 

(% runs) 

Caution 
Alert 
(% runs) 

Warning 
Alert 
(% runs) 

Aircraft A 

8 

30,156 

0.0 

0.1 

0.0 

10.5 

0.0 

0.0 

0.0 

0.0 

9 

25,848 

0.0 

0.0 

0.0 

1.9 

0.0 

0.0 

0.0 

0.0 

10 

17,232 

0.0 

0.0 

0.0 

0.5 

0.0 

0.0 

0.0 

0.0 

11 

12,924 

0.0 

0.0 

0.0 

0.2 

0.0 

0.0 

0.0 

0.0 

Truth 

4,308 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

Aircraft B 

8 

30,156 

0.3 

15.8 

0.0 

10.5 

4.0 

3.5 

0.0 

4.3 

9 

25,848 

0.5 

2.4 

0.0 

1.9 

0.0 

0.0 

0.0 

0.0 

10 

17,232 

0.3 

1.2 

0.0 

0.5 

0.0 

0.0 

0.0 

0.0 

11 

12,924 

0.1 

0.9 

0.0 

0.2 

0.0 

0.0 

0.0 

0.0 

Truth 

4,308 

0.0 

0.6 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 


exiting) was outside of the runway shoulder edges. 
As such, there was no buffer between when the 
aircraft was inside or outside the missed boundary so 
a measurable difference between the true and 
detected position could cause a missed boundary to 
be counted. 

The number of test runs that contained missed 
and nuisance indications and alerts were relatively 
low, overall, as shown in Table 11. The missed 
indication and alert definition, as noted previously, is 
algorithm dependent. Missed indications and alerts 
for both aircraft were highest when using NACp 8 
accuracy. For Aircraft A, only the number of missed 
warning alerts for NACp 8 accuracy was statistically 
different from NACp 9, 10, 11, and truth accuracies 
over all equipage levels. For Aircraft B, the number 
of nuisance TIs, missed and nuisance RSIs, and 
missed and nuisance warning alerts for NACp 8 
accuracy was statistically different from NACp 9, 10, 
11, and truth accuracies over all equipage levels. 

Aircraft B experienced missed RSIs under the 
truth accuracy condition. This was due to the 
transmission delay of the ADS-B model as described 
in the previous scenario. 

Nuisance indications and alerts were not issued 
on Aircraft A because the aircraft tracked the runway 
centerline during departure and liftoff. 


Unnecessary maneuvering - For the test runs in 
which both aircraft were broadcasting NACp 
accuracies and required to maneuver based on a 
warning alert, the percentage of test runs in which the 
aircraft maneuvered unnecessarily is shown in Table 
12. As the accuracy decreased, the frequency of 
occurrences of unnecessary maneuvers increased. 
With a NACp value of 8, approximately 8% or 8 in 
100 of the warning alerts were unnecessary. With a 
NACp value of 11, approximately 2 in 1000 warning 
alerts were unnecessary. The number of unnecessary 
maneuvers for NACp 8 accuracy was statistically 
different from NACp 9, 10, 11, and truth accuracies 
for both Aircraft A and Aircraft B . 


Table 12. Unnecessary Maneuvers by Aircraft 
during Departure / Taxi Crossing Runway 
Scenario 


NACp 

Aircraft A 
(% runs) 

Aircraft B 
(% runs) 

8 

8.5 

8.0 

9 

1.7 

1.4 

10 

0.5 

0.3 

11 

0.2 

0.2 


Collision avoidance - By the design of the 
scenarios, approximately 14% of the runs resulted in 
a near collision and approximately 6% resulted in a 
collision in the absence of CD&R, as shown in Table 





Table 13. Percentage of Near-Collisions (NC) and Collisions (C) for Equipage Combinations for 

Departure / Taxi Crossing Runway Scenario 


NACp 

# Runs per 
Equipage 

Neither 

Aircraft A 

Aircraft B 

Both 

NC 

C 

NC 

C 

NC 

C 

NC 

C 

8 

7,539 

14.2 

6.6 

12.7 

5.7 

13.8 

5.8 

12.5 

5.3 

9 

6,462 

14.2 

6.6 

12.6 

5.8 

14.1 

6.4 

12.6 

5.7 

10 

4,308 

14.2 

6.6 

12.6 

5.8 

14.0 

6.4 

12.6 

5.7 

11 

3,231 

14.2 

6.6 

12.6 

5.7 

14.0 

6.4 

12.5 

5.7 

Truth 

1,077 

14.2 

6.5 

12.6 

5.7 

14.0 

6.4 

12.6 

5.7 


13. The addition of CD&R only slightly improved 
collision avoidance with approximately 12% of the 
runs resulting in a near collision and approximately 
5% in a collision when both were equipped. Collision 
avoidance was not significantly affected by the 
CD&R system equipage levels. 

For the taxiing aircraft (Aircraft B), a warning 
alert generally did not occur until the aircraft was on 
the runway (i.e., mean position of Aircraft B was 
123.6 ft, SD 24.4, from the runway centerline when 
using truth accuracy). Since the aircraft was already 
past the runway shoulder, no action was taken and 
the aircraft continued across the runway. 

For the departing aircraft (Aircraft A) when 
using true position accuracy, 16.6% of the warning 
alerts occurred during takeoff roll when traveling 
between 50 and 80 kt, 41.9% occurred when 
traveling between 80 and 131 kt, and 41.5% occurred 
when traveling greater than 1 3 lkt. The aircraft was 
unable to abort the departure after reaching takeoff 
decision speed (131 kt); therefore, for 41.5% of the 
departures in which warning alerts were issued, the 
aircraft continued along its predefined departure path 
and collisions were unavoidable. As described 
above, alerts were not inhibited above 80 kt as 
specified in the SURF IA SPR. 

Position accuracy had little effect on collision 
avoidance; however, there were fewer collisions 
when Aircraft B was equipped when using NACp 8 
accuracy. As in the previous scenario, this was 
because, in some instances, Aircraft B was able to 
conduct emergency braking and stop before reaching 
the runway shoulder. Even though the warning alert 
was generated when the aircraft was already on the 
runway (according to the data), the actual location of 
the aircraft was far enough back from the runway so 


it could come to a complete stop before actually 
reaching the runway shoulder. Over all levels of 
NACp accuracy, the differences in the number of 
collisions showed no statistically significant 
difference. 

Summary 

The SURF IA CD&R algorithm was evaluated 
in a fast-time batch simulation study. The purpose of 
the study was to evaluate the performance of the 
aircraft-based SURF IA algorithm during various 
runway scenarios, multiple levels of CD&R system 
equipage, and various levels horizontal position 
accuracy. 

It is critical that alerting does not cause the flight 
crew to make unnecessary maneuvers since these 
actions can cause delays, equipment wear, and other 
costs to airlines. Results showed that for the scenarios 
reported on herein, maneuvers were frequently made 
unnecessarily when using NACp 8 position accuracy. 

Alert toggling occurs when multiple instances of 
indications or alerts are generated as a result of 
position accuracy or aircraft maneuvering. Alert 
toggling can be a distraction to the flight crew and 
could cause mistrust in the technology. In general, 
alert toggling occurred more frequently as the 
position accuracy was reduced, especially for NACp 
8 and NACp 9 accuracy levels. 

Missed indications and alerts represent a 
reduction in CD&R benefits and will result in 
operations such as they currently exist, where CD&R 
indications and alerts are not provided. The 
occurrence of missed indications and alerts was much 
higher with NACp 8 position accuracy, in general. 




Nuisance indications and alerts could distract the 
flight crew unnecessarily, reduce confidence in the 
system, and can negatively affect safety and 
operational effectiveness. For the test runs in which 
nuisance indications and alerts were possible, 
nuisance indications and alerts only occurred when 
using NACp 8 position accuracy. 

For the scenarios reported on herein, the CD&R 
equipage level did not have a major impact on 
collision avoidance. More collisions were avoided if 
both aircraft were equipped with CD&R, but 
generally, the aircraft taxing across the runway was 
not issued a warning alert until the aircraft was on the 
runway and, therefore, did not take action and 
continued across the runway. Positional accuracy had 
surprisingly less effect on collision avoidance 
effectiveness than previously assumed. Continued 
analysis is necessary to understand if the ineffective 
CD&R was due to a test set-up assumption or is, in 
fact, a design fault of the present SURF IA SPR. 
Also, analysis is required to better understand the 
SPR 80 kt restriction requirement for alerting during 
departure, the impact of not inhibiting alerts above 80 
kt during departure, and the trade-off between missed 
alerts and increased nuisance/unnecessary alerting. 

Continued analysis of these data is being 
conducted to identify horizontal positional accuracy 
requirements for effective terminal maneuvering area 
CD&R and efficient algorithm designs. 
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